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EEG  Correlates  of  Fluctuation  in  Cognitive  Performance 
in  an  Air  Traffic  Control  Task 


INTRODUCTION 

Mental  fatigue,  defined  as  the  inability  of  the  human  brain  to 
allocate  sufficient  resources  to  perform  a  task  at  normal  capacity 
(Smit,  Eling,  &  Coenen,  2004),  can  impair  a  variety  of  cognitive 
functions  that  are  vital  in  performing  a  task  effectively  (Bills, 
1931;  Lorist,  Klein,  Nieuwenhuis,  De  Jong,  Mulder,  &  Meijman, 
2000;  Sanders  &  Hoogenboom,  1970).  Hence,  it  is  desirable 
to  study  the  effects  of  mental  fatigue  and  its  impact  on  human 
performance  and  work  routines.  Mental  fatigue  that  happens  after 
hours  of  work,  known  as  the  time-on-task  effect,  can  significantly 
impact  vigilance  (Mast  &  Heimstra,  1964;  Smit  et  ah,  2004), 
problem  solving  capability  (Horne,  1 988;  Van  der  Linden,  Frese, 
&  Meijman,  2003)  and  situational  awareness  (Vidulich,  Stratton, 
Crabtree,  &  Wilson,  1994),  which  are  the  basis  for  completing 
most  real-world  tasks  successfully.  Mental  fatigue  is  of  special 
concern  for  air  traffic  control  specialists  (ATCSs)  due  to  their 
cognitively  demanding  workload  and  public  safety  factors.  Air 
traffic  control  tasks  require  ATCSs  to  maintain  a  constant  focus 
with  high  reliability  and  to  use  multitasking  capabilities  to  pro¬ 
cess  information  from  a  variety  of  sources  for  considerably  long 
durations  (Kallus,  Van  Damme,  &  Dittmann,  1 999) .  ATCSs  stay 
in  the  forefront  of  air  traffic  flow  control  and  communicate  vital 
information  such  as  the  proximity  of  other  aircraft,  the  impact 
of  weather,  and  the  status  of  runways.  These  tasks  require  high 
levels  of  cognitive  ability  and  endurance  to  complete.  Performing 
these  tasks  for  long  hours  can  potentially  result  in  mental  fatigue 
that  could  be  a  hindrance  to  task  completion.  Since  these  tasks 
require  high  reliability  to  keep  the  passengers  on  board  safe, 
understanding  how  controllers’  mental  functions  are  affected  is 
essential  to  maintaining  safety. 

Modern  physiological  measures  such  as  electrocardiography 
(ECG),  respiration,  eye  blinks,  and  electroencephalography 
(EEG)  have  been  proven  more  reliable  and  effective  in  monitor¬ 
ing  fatigue  (Brookings,  Wilson,  &  Swain,  1996;  Han,  Wang, 
L.,  Wang,  P.,  &  Wen,  2005)  in  real  time  rather  than  subjective 
measures.  Among  these  physiological  measures,  EEG  has  been 
used  more  extensively  in  recent  times  due  to  its  non-invasive 
nature,  high  reliability  (Lai,  Craig,  Boord,  Kirkup,  &  Nguyen, 
2003;  Murata,  Uetake,  &Takasawa,  2005),  and  reproducibility 
(Lai  &  Craig,  2005).  EEG  power  spectral  measures  are  reported 
to  produce  patterns  that  can  be  used  to  identify  changes  in  alert¬ 
ness  (Makeig  &  Inlow,  1993;  Makeig  &  Jung,  1995). 

Researchers  have  identified  EEG  fluctuations  correlated  to 
performance  deficits  (Jung,  Makeig,  Stensmo,  &  Sejnowski,  1 997; 
Kilmesch,  1999;  Kilmesh,  Doppelmayr,  Russegger,  Pachinger,  & 
Schwaiger,  1998;  Lai  and  Craig,  2002,  2005;  Makeig  &  Inlow, 
1993;  Makeig  &  Jung,  1995).  Event-related  potentials  (ERPs), 
small  blocks  of  mean  EEG  phase-locked  to  the  onset  of  stimuli, 


have  shown  significant  patterns  corresponding  to  performance 
change  and  mental  state  shift  (De  Lugt,  Loewy,  &  Campbell, 
1996;  Pfurtscheller,  1992;  Pfurtscheller  &  Aranibar,  1977; 
Neuper  &  Pfurtscheller,  2001).  Another  way  of  identifying 
EEG  patterns  is  by  using  time-frequency  analysis.  This  method 
has  shown  significant  increases  in  alpha  and  theta  powers,  time- 
locked  to  stimulus  presentation  (Kilmesch,  1999;  Akerstedt  & 
Gillberg,  1990;  Torsvall  &  Akerstedt,  1987). 

Most  of  these  research  studies  on  mental  fatigue  use  either 
event-driven  hour-long  tasks  (Huang,  Jung,  Delorme,  &  Makeig, 
2008;  Jap,  Lai,  Fischer,  &  Bekiaris,  2009)  or  considerably  long- 
duration  tasks,  focused  on  effects  of  monotony  or  workload 
changes  from  EEG  data  analysis.  Only  a  very  few  studies  have 
focused  on  determining  the  effects  of  mental  fatigue  and  physi¬ 
ological  markers  for  mental  fatigue  in  real-world  work  scenarios. 
In  our  research,  we  approached  the  problem  of  mental  fatigue 
by  correlating  the  performance  changes  with  EEG  patterns 
that  represent  the  development  of  mental  fatigue  over  time. 
We  conducted  four  experimental  sessions  using  simulation 
tasks  from  a  low-fidelity  air  traffic  control  software  package, 
i.e.,  C-Team  (Bailey,  Broach,  Thompson,  &  Enos,  1999),  with 
increasing  work  time,  ranging  from  one  half  hour  to  two  hours, 
for  determining  the  amount  of  change  in  performance  and  EEG 
patterns  associated  with  mental  fatigue  over  time.  This  provided 
a  better  means  to  understand  the  effects  of  fatigue  over  different 
intervals  of  time  and  their  effect  on  brain  functions. 

Furthermore,  it  is  essential  to  determine  the  relationship 
between  cognitive  performance  changes  and  EEG  patterns 
indicative  of  mental  fatigue  to  develop  a  reliable  method  for 
detecting  time-on-task  fatigue  in  real  time.  To  achieve  this,  we 
hypothesized  that  a  prominent  change  in  EEG  spectral  power 
can  be  a  marker  for  change  in  mental  state,  as  reported  by  other 
studies  (Gevins  et  ah,  1995;  Wilson  &  Russell,  2003).  Since  our 
method  provides  a  means  for  continuous  evaluation,  our  findings 
can  be  valuable  in  the  design  of  continuous  fatigue  monitoring 
models,  in  contrast  to  earlier  neural  network  models  focused  on 
fatigue  prediction  based  on  ERPs  (Murata  et  ah,  2005). 

In  summary,  we  conducted  a  study  to  investigate  continuous 
EEG  spectral  power  changes  that  are  correlated  with  perfor¬ 
mance  change  during  a  low-fidelity  air  traffic  control  task.  To 
determine  EEG  spectral  power  changes,  we  conducted  a  power 
spectral  analysis  and  implemented  a  non-parametric  statistical 
approach  on  the  EEG  data  collected  from  four  experimental 
sessions  (30,  60,  90,  &  1 20  minutes) .  Next,  we  computed  differ¬ 
ent  performance  measures  such  as  response  time,  routing  time, 
number  of  crashes,  and  number  of  warnings  from  the  C-Team 
data.  A  correlation  between  EEG  spectral  power  changes  and 
performance  data  was  then  established  by  comparing  significant 
changes  observed  in  these  data  and  their  occurrences  over  time. 
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METHOD 

Participants 

We  recruited  1 0  healthy  male  participants  from  the  population 
of  students  at  the  University  of  Oklahoma  between  the  ages  of  22 
and  30  (Mean±SD  =  25 ±2.3,  all  right  handed)  having  no  prior 
experience  with  air  traffic  control.  All  participants  took  part  in 
four  recording  sessions  that  lasted  for  one  half-hour,  one  hour, 
an  hour  and  a  half,  and  two  hours,  which  will  be  referred  to  as 
Session  1 ,  Session  2,  Session  3,  and  Session  4,  respectively,  totaling 
40  data  collection  sessions.  However,  data  from  Participant  1 
(Session  3)  and  Participant  6  (Session  3)  were  excluded  due  to 
high  noise  levels  that  could  not  be  improved  by  pre-processing 
using  filtering  and  artifact  rejections.  All  participants  took  part 
in  the  study  after  giving  informed  consent  according  to  the 
University  of  Oklahoma  Institutional  Review  Board  standards. 
Participants  were  compensated  monetarily  for  their  time. 

Experimental  Protocol 

The  Minicog  Rapid  Assessment  Battery  (MRAB)  (Shephard, 
Kho,  Chen,  &  Kossyln,  2006)  and  the  C-Team  low-fidelity  air 
traffic  control  simulator  (Bailey  et  ah,  1999)  were  used  to  study 
behavioral  changes  associated  with  the  time-on-task  effect  (or 
mental  fatigue).  Participants  performed  MRAB  cognitive  tests 
before  and  after  recording  sessions  to  identify  cognitive  perfor¬ 
mance  changes  associated  with  time  on  task.  The  MRAB  had 
measures  of  seven  cognitive  tasks,  including  Vigilance,  Filtering, 
Divided  Attention,  Mental  Rotation,  Working  Memory,  Cogni¬ 
tive  Set  Switching,  and  Perceptual  Reaction. 

Vigilance,  filtering,  and  divided  attention  tasks  were  used  to 
test  the  ability  of  a  participant  to  focus  on  specific  information 
in  order  to  conduct  a  task  efficiently.  The  Vigilance  task  assesses 
the  ability  to  maintain  continuous  attention  to  specific  events, 
the  Filtering  task  assesses  the  ability  to  filter  out  unwanted  in¬ 


formation,  and  the  Divided  Attention  task  assesses  the  ability 
to  maintain  attention  while  processing  two  different  types  of 
information.  Working  Memory  investigates  the  ability  to  retain 
information  and  respond  to  an  event  based  on  information  from 
previous  events.  Cognitive  Set  Switching  assesses  the  ability 
to  switch  among  cognitive  sets  to  process  suddenly  appearing 
stimuli  in  a  sequence  displaying  the  same  type  of  information. 
Mental  Rotation  requires  forming  judgments  about  two  geo¬ 
metrical  shapes  to  determine  whether  they  are  mirrored  images 
or  identical  images  with  rotation.  Finally,  the  Perceptual  Reac¬ 
tion  Time  task  is  used  to  check  on  quickness  and  accuracy  as 
the  stimulus  shows  up  at  different  locations  and  corresponding 
keys  must  be  pushed. 

Performance  data  recorded  by  the  C-Team  task  were  used  for 
the  continuous  evaluation  of  fatigue  effects  with  time  on  task. 
The  C-Team  task  used  for  the  present  study  is  shown  in  Figure 
1  (a).  C-Team  was  run  on  a  PC  with  Windows  operating  system, 
and  the  operational  interface  was  displayed  on  a  22-inch  monitor. 
The  participants  clicked  on  appropriate  controller  tabs  using  a 
pointing  device  (i.e.,  computer  mouse)  to  control  aircraft.  The 
participants  were  required  to  activate  and  navigate  aircraft  that 
appeared  on  the  display  screen  at  the  rate  of  three  per  minute. 
Their  goal  was  to  get  aircraft  to  their  respective  airports  or  exit 
gates,  as  defined  on  each  aircraft  data  block.  The  number  of 
aircraft  requiring  activation  was  maintained  at  a  constant  rate 
for  the  entire  task.  The  scenarios  were  developed  in  such  a  way 
that  airport  orientation  changed  every  five  minutes  with  a  no¬ 
tification  that  occurred  one  minute  prior  to  the  change  through 
auditory  feedback.  Proximity  warnings  were  issued  whenever 
aircraft  got  too  close  to  the  boundary  edges,  airport  edges,  other 
aircraft,  and  restricted  areas.  Failure  to  respond  to  these  warnings 
resulted  in  a  crash. 

Figure  1  (a)  shows  the  C-Team  task  display  screen,  with  the 
aircraft  heading  to  the  exit  B  along  the  routing  path,  represented 


Figure  1  (a).  C-Team  task  display  screen  showing  the  aircraft  (green  arrow),  airports  (green 
circles  with  center  runway),  restricted  area  (red  circle)  and  the  route  traversed  (red  line)  to 
reach  the  exit  B.  (b)  Accumulated  routing  paths,  representing  the  task  complexity,  and  the 
trajectory  used  by  participants  for  air  traffic  control. 
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by  the  red  line.  The  delay  encountered  in  activating  an  aircraft 
upon  its  appearance  on  the  C-Team  display  screen  is  called 
response  time.  Routing  time,  the  time  taken  for  the  aircraft  to 
reach  the  exit  B,  is  defined  as  the  amount  of  time  between  the 
appearance  of  aircraft  on  the  screen  and  final  time  to  reach  the 
target  exit/airport.  Figure  1(b)  shows  the  accumulated  routing 
paths  of  all  aircraft  from  Participant  2,  Session  4.  These  path 
maps  show  the  complexity  in  routing  introduced  by  the  central 
restricted  area  and  the  changes  in  aircraft  orientation,  which  are 
indicated  by  the  accumulated  red  lines  around  the  airport  in  all 
four  directions.  Upon  completion  of  the  task,  the  performance 
data  were  saved  to  a  replay  file  for  further  processing  during  the 
data  analysis  procedures. 

The  experimental  protocol  had  two  phases:  training  and 
recording.  During  the  training  phase,  participants  were  briefed 
on  the  use  of  the  MRAB  software  and  performed  four  MRAB 
practice  sessions  on  different  days  within  a  week.  After  the  MRAB 
practice  sessions,  participants  underwent  three  training  sessions 
of  C-Team  to  get  them  acquainted  with  the  task  requirements 
and  to  develop  their  own  strategies  for  controlling  aircraft.  In 
the  recording  phase,  participants  took  part  in  four  sessions, 
each  of  which  was  conducted  on  different  days.  Each  recording 
session  consisted  of  initial  MRAB  test,  followed  by  the  C-Team 
air  traffic  control  task,  with  simultaneous  EEG  recording,  and 
final  MRAB  test.  Four  C-Team  sessions  varied  by  30  min  (30 
min,  60  min,  90  min  and  120  min). 

EEG  Data  Acquisition 

EEG  data  were  collected  using  a  128-channel,  high-density 
system  at  a  sampling  frequency  of  250  Hz.  During  recording 
sessions,  participants  were  advised  to  minimize  facial  move¬ 
ments  and  eye  blinks  to  reduce  artifacts.  An  impedance  check 
for  the  electrode  cap  was  performed  at  60-minute  intervals 
during  sessions  3  and  4  to  keep  the  impedance  lower  than  50 
KO  to  ensure  high  signal-noise-ratio  (SNR)  in  EEG  data.  The 
impedance  check  lasted  for  5  to  10  minutes  in  all  participants. 

Performance  Data  Analysis 

The  performance  data  obtained  from  the  C-Team  replay 
files  and  MRAB  log  files  were  backed  up  after  the  successful 
completion  of  each  session.  Response  times  and  errors  were 
computed  for  MRAB  tasks  that  were  performed  before  and  after 
the  C-Team  task.  We  statistically  compared  response  times  and 
numbers  of  errors  from  pre-task  and  post-task  MRAB  tests.  To 
further  evaluate  the  individual  cognitive  test  performance  in 
MRAB,  we  established  a  baseline  performance,  averaged  from 
the  MRAB  data  obtained  during  three  C-Team  training  sessions, 
which  showed  stable  performance  after  the  initial  eight  MRAB 
practice  sessions.  We  then  compared  the  performance  of  MRAB 


tasks  from  the  recording  sessions  with  the  established  baseline  to 
study  the  effects  of  fatigue  associated  with  time  on  task. 

C-Team  data  were  obtained  from  the  replay  files  to  compute 
performance  measures  such  as  response  time,  routing  time, 
number  of  crashes,  and  number  of  warnings.  The  crash  count 
was  a  composite  of  the  number  of  mid-air  collisions,  not  be¬ 
ing  lined  up  with  the  runway  at  the  time  of  landing,  colliding 
with  boundaries  of  C-Team  airspace,  and  entering  a  section  of 
restricted  air  space.  A  proximity  warning  was  issued  prior  to  a 
crash.  The  C-Team  data  were  segmented  into  1 0-minute  inter¬ 
vals,  and  average  values  of  response  time  and  routing  time  were 
computed  within  each  interval.  Parameters  such  as  crashes  and 
proximity  warnings  were  also  computed  for  the  respective  seg¬ 
ments.  A  reference  segment  with  the  best  performance  within 
the  first  50  minutes  of  the  session  was  identified  as  the  baseline 
segment.  Two-tail  £-tests,  using  a  significance  level  of p  <  0.05 
compared  to  the  baseline  segment,  were  then  performed  to 
identify  the  segments  showing  significant  change  in  response 
time  and  routing  time  for  Sessions  3  and  4. 

EEG  Data  Analysis 

EEG  data  were  manually  inspected  for  channels  with  higher 
voltage  fluctuations  over  the  course  of  recording  and  were 
marked  as  bad  channels.  EEG  data  on  the  marked  bad  channels 
were  then  interpolated  from  neighboring  channels  through  the 
bad  channel  replacement  tool  in  EGI  net  station  4.2  (Electrical 
Geodesics,  Inc.,  Eugene,  OR).  The  procedure  was  reasonable 
since  the  electrical  field  in  the  neighboring  areas  usually  had 
continuous  distribution  due  to  the  volume  conductance  effect. 
After  bad  channel  replacement,  a  band-pass  filter  of  0.5  -  100 
Hz  was  applied  to  the  data. 

The  C-Team  task  required  frequent  hand  movements  due 
to  the  use  of  a  computer  mouse  to  perform  control  actions. 
This  was  accompanied  by  frequent  head  movements,  which 
created  artifacts  in  EEG  data  (i.e.,  electromyography,  EMG). 
These  artifacts  usually  showed  temporary  high-amplitude,  high- 
frequency  components,  and  many  EMG  signals  had  localized 
spatial  distributions  over  the  scalp.  The  artifacted  EEG  data  were 
identified  by  visual  inspection  and  removed  using  the  Fieldtrip 
software  (Oostenveld,  Maris,  &  Schoffelen,  2011). 

EEG  data  were  then  segmented  into  1 0-minute  intervals  on 
which  further  analyses  were  carried  out.  To  remove  the  artifacts 
introduced  by  the  impedance  check  on  Sessions  3  and  4  data, 
we  performed  an  independent  component  analysis  (ICA)  using 
EEGLAB  software  (Delorme  &  Makeig,  2004)  on  continu¬ 
ous  EEG  data  before  segmentation.  During  this  process,  the 
EEG  data  were  applied  with  an  extended  infomax  ICA,  using 
Runica  (Makeig,  Jung,  Bell,  &  Sejnowski,  1996)  from  the 
EEGLAB  toolbox. 
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Figure  2  (a).  Identified  artifact  components  after  running  ADJUST  artifact  rejection  method  in  EEGLAB. 


We  set  the  number  of  independent  components  (ICs)  to  64  and  used  the  ADJUST  (Mognon,  Jovicich,  Bruz- 
zone,  &  Buiatti,  2010)  plugin  from  EEGLAB  tool  to  remove  artifacts  related  to  eye  movements  and  generic 
discontinuities  (Figure  2(a)). 
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Figure  2  (b).  Impedance  check  induced  artifact  removal  based  on  the  visual  inspection  of  power  spectral 
plots.  The  components  impacted  by  impedance  check  that  were  identified  and  removed  are  marked  using 
red  boxes. 


Components  affected  by  the  impedance  check  were  visually 
identified  based  on  the  power  spectral  density  (PSD)  of  each  (in 
the  Whisker  plots  of  data  from  10-minute  segments),  as  shown 
in  Figure  2(b).  The  artifacted  components  are  indicated  by  red 
boxes.  By  removing  these  artifact  ICs,  the  “artifact-free”  EEG 
data  were  reconstructed  from  the  residual  EEG  data. 

The  above  processed  data  were  sectioned  into  1  -second  ep¬ 
ochs,  and  a  short-time  window  Fast  Fourier  Transform  (FFT) 
was  applied  with  a  frequency  resolution  of  1  Hz  to  obtain 
spectral  powers  of  three  frequency  components,  which  were 
theta  (4  Hz  -  8  Hz),  alpha  (8  Hz  -  12  Hz),  and  beta  (12  Hz  -  30 
Hz)  components  (Figure  3).  To  determine  the  time  frequency 
representations  using  FFT,  we  utilized  a  multi-taper  approach 
(Percival  &  Walden,  1993)  with  fixed  window  size  of  0.25s  and 
50%  overlap.  Segments  were  windowed  using  a  Hanning  window 
(Blackman  &  Tukey,  1959),  and  power  spectra  were  computed 
and  averaged  over  the  resulting  eight  equal  segments  in  1  second. 

Non-Parametric  Statistical  Analysis 

To  identify  significant  changes  in  mental  state  due  to  time  on 
task,  a  non-parametric  statistical  method  (Maris  &  Oostenveld, 
2007)  was  used.  This  method  combines  the  merits  of  statistical 
testing  and  non-parametric  techniques  to  handle  the  multiple 
comparison  problems  in  EEG  data.  In  this  analysis,  we  statisti¬ 


cally  compared  the  first  1 0-minute  segment  with  the  remaining 
1 0-minute  segments  over  time.  The  comparison  study  resulted  in  a 
sequence  of  maps  that  illustrated  the  EEG  spectral  power  changes 
as  a  function  of  time.  The  ^-statistic  value  for  each  comparison 
was  computed  and  the  threshold  was  set  at  an  alpha  value  of 
0.05.  lvalues  from  all  EEG  channels  were  grouped  into  clusters 
based  on  the  spatial  adjacency  of  electrodes.  The  cluster-level 
test  statistic  was  computed  as  the  sum  of  ^-values  from  a  cluster. 
Then  we  used  the  non-parametric  method  to  find  the  significant 
clusters  using  a  permutation  procedure.  The  procedure  involved 
combining  data  from  two  1 0-minute  segments  (which  are  com¬ 
pared)  into  a  single  dataset  and  then  randomly  drawing  samples 
from  the  combined  dataset  to  form  two  new  groups  of  data  of  the 
original  size.  This  procedure  is  known  as  random  partitioning. 
The  cluster-level  test  statistic  was  then  computed  for  these  two 
randomized  partitions  in  the  same  manner  as  for  the  original 
1 0-minute  segment  data.  Repeating  this  random  partitioning  an 
infinite  number  of  times  results  in  a  permutation  distribution 
in  which  the  p-v alue  is  called  the  permutation  p-v alue.  Due  to 
practical  constraints,  the  permutation y>-value  was  obtained  using 
a  Monte  Carlo  estimate,  which  was  obtained  by  repeating  the 
random  partitioning  a  large  number  of  times  (e.g.,  1,000).  A 
permutation  />-value  was  obtained  that  was  the  proportion  of  a 
random  partition  that  had  an  observed  cluster-level  test  statistic 
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Figure  3.  Computation  of  EEG  PSD.  (a)  10-second  EEG  signals  from  one  channel  from  recordings;  (b) 
EEG  signals  in  a  resolution  of  one  second  used  to  perform  the  FFT  calculation;  (c)  One-second  EEG 
signals  multiplied  by  a  sequence  of  Hanning  windows  each  of  0.25-second  length  and  50%  overlap;  (d) 
Resulted  eight  periodic  tapers  from  the  operation  of  Hanning,  windows;  (e)  PSD  from  each  taper;  (f)  EEG 
PSD  obtained  by  averaging  PSDs  from  eight  tapers. 
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greater  than  the  cluster-level  test  statistic  from 
original  segment  data.  If  this  Rvalue  was  less 
than  the  set  critical  alpha  value  (<  0.05),  we 
concluded  that  the  two  1 0-minute  segments 
were  significantly  different. 

The  above-mentioned  non-parametric 
statistical  method  was  implemented  in  the 
Fieldtrip  software  (Maris  &  Oostenveld, 
2007),  and  its  graphic  illustration  is  displayed 
in  Figure  4.  A  ^-statistic  was  then  plotted  out 
at  each  electrode  indicated  by  three  shades  of 
colors  to  illustrate  the  change  in  power  spectra 
over  human  heads  as  a  function  of  time.  A  blue 
color  represented  a  significant  increase,  while 
a  red  color  indicated  a  significant  decrease.  A 
green  color  indicated  no  change  in  the  power 
spectra  compared  to  the  first  1 0-minute  seg¬ 
mented  data  (see  Fig.  4  for  examples).  The 
significant  change  in  EEG  spectral  powers, 
identified  by  the  implementation  of  the  sta¬ 
tistical  test,  indicated  a  possible  mental  state 
transition,  as  discussed  below. 

Mental  State  Analysis 

Mental  state  was  determined  based  on 
the  activation  of  the  neural  networks,  which 
were  responsible  for  the  neural  computation 
performed  by  the  human  brain.  For  example, 
mental  states  have  been  studied  as  microstate 
and  transition  between  different  microstates  as 
an  indicator  of  a  functional  status  change  in 
the  human  brain  (Pascual-Marqui,  Michel,  & 
Lehmann,  1995).  Such  transitions  of  mental 
states,  or  microstates,  are  reflected  in  rhythmic 
EEG  signals.  In  other  words,  EEG  measures 
the  signal  produced  by  the  synchronous 
discharging  population  of  cortical  neurons, 
which  are  shifted  under  different  mental  states. 

To  determine  the  change  in  brain  activity, 
we  selected  the  channels  corresponding  to 
the  significant  change  from  non-parametric 
analyses  and  searched  for  the  change  in  the 
absolute  spectral  power  over  time  in  a  specific 
frequency  band  (i.e.,  the  theta,  alpha,  and  beta 
bands).  Specifically,  the  pattern  in  the  median 
of  EEG  spectral  powers  from  10-minute  seg¬ 
ment  data  along  the  time  axis  (i.e.,  0.5,  1, 
1.5,  or  2  hours)  was  assessed.  This  indicated 
the  mean  level  of  activity  at  each  frequency 
band.  We  then  searched  for  pattern  shifts  in 
the  curves  for  medians  as  indicators  of  the 
mental  state  transition,  as  studied  by  Pascual- 
Marqui  et  ah,  1995.  We  hypothesized  that  the 
prominent  shift  in  EEG  spectral  power  data 
was  correlated  to  the  time-on-task  effect.  We 
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Figure  4.  Illustration  of  cluster-based  permutation  test  for  spectral  powers 
on  channels  between  two  10-minute  EEG  data. 
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also  evaluated  these  features  using  other  types  of  data,  such  as  the 
performance  data  (see  the  section  below) .  The  time  exhibiting  the 
significant  prominent  shift  was  noted  as  the  transition  time  (TT) . 

Correlation  Analysis 

To  find  the  association  between  the  EEG  patterns  indicative 
of  mental  fatigue  and  cognitive  performance,  we  compared 
performance  measures  such  as  response  time  and  routing  time 
with  EEG  spectral  power  data  over  time.  Response  time  and 
routing  time  represented  the  change  in  performance,  while 
EEG  spectral  power  represented  the  change  in  mental  state  due 
to  the  time-on-task  effect.  We  compared  the  results  obtained 
from  non-parametric  statistical  testing  to  identify  the  change  in 
spatio-temporal  patterns  of  EEG  spectral  powers  and  changes 
in  C-Team  performance  measures  over  time. 


RESULTS 

MRAB  Results 

Overall  results  from  the  MRAB  tasks  showed  stable  perfor¬ 
mance  with  no  significant  changes  in  reaction  time  both  before 
and  after  sessions  of  any  length  (i.e.,  0.5,  1,  1.5,  and  2  hours). 
The  comparison  study  against  baseline  performance  using  t-tests 
showed  similar  performance  between  pre-task  and  post- task  tests, 
with  the  exception  that  there  were  observed  differences  for  the 
Vigilance  and  Filtering  tasks.  However,  these  differences  between 
pre-task  and  post-task  were  not  consistent  among  participants 
at  the  level  of  group  analysis.  Hence,  no  conclusive  evidence 
of  time-on-task  effects  was  indicated  by  the  performance  data 
from  the  MRAB  tasks.  Meanwhile,  the  performance  measures 
obtained  from  C-Team  showed  significant  effects  of  time  on 
task,  as  discussed  below. 

C-Team  Results 

C-Team  performance  data  were  used  to  evaluate  the  time- 
on-task  effects  during  the  task  scenarios.  In  general,  when  time 
on  task  became  longer,  all  participants  had  increased  response 
times  in  activating  aircraft  and  increased  routing  times  to  reach 
destinations.  It  appeared  that  it  took  about  20  to  30  minutes 
for  participants  to  reach  stable  performance  after  beginning  the 
task.  This  result  was  also  suggested  by  the  EEG  data  and  will  be 
discussed  in  the  next  section. 
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Figure  5  C.  Team  performance  measures,  reaction  time,  and  routing  time  are 
plotted  over  time  to  show  the  significantly  deteriorating  performance  happening 
around  60  -  80  minutes  for  session  2  (blue),  session  3  (red),  and  session  4 
(green)  from  participant  3. 


Figure  5  shows  the  response  times  and  routing  times  from  sessions  2,  3,  and  4  of 
Participant  3.  During  the  first  60  minutes,  response  times  and  routing  times  followed  a 
similar  pattern  for  all  three  sessions.  That  is,  there  was  an  initial  drop  in  times,  followed  by 
a  gradual  increase  toward  the  60-minute  mark.  Following  the  60-minute  mark,  significant 
changes  were  observed  in  Sessions  3  and  4  response  times  and  routing  times.  These  changes 
were  evident  for  all  participants. 
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Table  1.  Response  time  significance  level  between  reference  interval  (C*)  and  rest  of  the  time  intervals 
(T#)  obtained  from  C-Team  task  of  all  participants  (P)  for  session  4. 


Time  Intervals 


c* 

T3 

T4 

T5 

T6 

T7 

T8 

T9 

T10 

Til 

T12 

PI 

T4 

0.372 

0.117 

0.074 

0.131 

0.368 

0.775 

0.108 

0.294 

P2 

T4 

- 

- 

0.320 

0.220 

0.009 

0.089 

0.385 

0.150 

0.699 

- 

P3 

T5 

- 

- 

- 

0.023 

0.000 

0.011 

0.107 

0.000 

0.107 

0.506 

P4 

T2 

0.130 

0.021 

0.486 

0.119 

0.016 

0.095 

0.079 

0.111 

0.023 

0.793 

P5 

T5 

- 

- 

- 

0.065 

0.084 

0.000 

0.069 

0.004 

0.861 

- 

P6 

T5 

- 

- 

- 

0.161 

0.010 

0.702 

0.009 

0.298 

0.154 

0.286 

P7 

T2 

0.358 

0.217 

0.462 

0.053 

0.013 

0.025 

0.634 

0.104 

0.094 

0.696 

P8 

T2 

0.513 

0.083 

0.632 

0.068 

0.244 

0.014 

0.261 

0.160 

0.155 

0.502 

P9 

T4 

- 

- 

0.770 

0.291 

0.291 

0.160 

0.536 

0.210 

0.613 

0.003 

P10 

T2 

0.457 

0.389 

0.445 

0.344 

0.781 

0.415 

0.520 

0.120 

0.509 

0.002 

Note:  The  values  italicized  and  underlined  are  identified  to  be  significantly  increasing  ones  based  on  the  t-test 
conducted  with  significance  level  of  P  <  0.05. 


Table  2.  Routing  time  significance  level  between  reference  interval  (C*)  and  rest  of  the  time  intervals  (T#) 
obtained  from  C-Team  task  of  all  participants  (P)  for  session  4. 


Time  Interval 


C* 

T3 

T4 

T5 

T6 

T7 

T8 

T9 

T10 

Til 

T12 

PI 

T4 

- 

- 

0.795 

0.183 

0.009 

0.118 

0.205 

0.466 

0.253 

0.580 

P2 

T5 

- 

- 

- 

0.591 

0.201 

0.178 

0.861 

0.672 

0.661 

0.260 

P3 

T5 

- 

- 

- 

0.153 

0.000 

0.199 

0.222 

0.011 

0.512 

0.549 

P4 

T5 

- 

- 

- 

0.578 

0.349 

0.692 

0.429 

0.913 

0.784 

0.080 

P5 

T5 

- 

- 

- 

0.521 

0.729 

0.054 

0.553 

0.706 

0.601 

0.381 

P6 

T5 

- 

- 

- 

0.336 

0.006 

0.044 

0.084 

0.905 

0.071 

0.798 

P7 

T5 

- 

- 

- 

0.591 

0.178 

0.135 

0.980 

0.817 

0.323 

0.315 

P8 

T5 

- 

- 

- 

0.628 

0.433 

0.108 

0.788 

0.466 

0.410 

0.698 

P9 

T4 

- 

- 

0.600 

0.310 

0.450 

0.136 

0.792 

0.610 

0.904 

0.377 

P10 

T1 

0.270 

0.052 

0.008 

0.013 

0.130 

0.093 

0.632 

0.251 

0.726 

0.326 

Note:  the  values  italicized  and  underlined  are  identified  to  be  significantly  increasing  ones  based  on  the 
t-test  conducted  with  significance  level  of  P  <  0.05. 


Tables  1  and  2  present  the  statistical  results  via  comparing 
10-minute  segment  data  with  the  baseline  segment  data  for 
Sessions  3  and  4,  respectively.  As  shown  by  the  £-test  results,  we 
inferred  that  there  was  a  significant  increase  in  response  time  and 
routing  time  that  occurred  approximately  between  60  and  80 


minutes.  Following  the  60-80  minute  region,  some  participants, 
such  as  Participant  3,  exhibited  momentary  recovery  in  terms  of 
performance,  followed  by  a  significant  decline  in  performance.  It  is 
possible  that  these  participants  noticed  their  performance  decline 
and  were  able  to  temporarily  return  to  baseline  performance. 
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Table  3.  Summary  of  C-Team  task  performance  measures  for  all  the  sessions. 


Mean 

Response  time 

Routing  time 

Crashes 

Warnings 

Active  plane/second 

Session  1 

4.26 

29.72 

1.40 

2.00 

1.25 

Session  2 

5.08 

31.56 

3.67 

3.33 

1.31 

Session  3 

5.78 

31.23 

6.67 

9.80 

1.24 

Session  4 

4.91 

30.31 

4.50 

6.80 

1.26 

rivsT2  TlvsT3  Tl  vs  14  TIvsiTS  TlvsTfc  TlvsT7  TiusTS 


TlvsTi  TlvsT3  Tl  vs  T4  TlvsTS  IlvsTS  TlvsT?  Tlv$T8 

Participant#  8 


Figure  6.  Scalp  maps  plotted  out  from  the  non-parametric  statistical  test  results  showing  significant 
changes  in  EEG  power  spectra  across  alpha  (a),  beta  (b)  and  theta  (c)  over  time,  comparing  the  first  10- 
minute  segment  with  subsequent  segments,  and  resulted  significance  levels  are  color  coded  and 
displayed.  Red  indicates  a  significant  decrease,  blue  indicates  a  significant  increase,  and  green  means 
no  significant  change. 


Performance  decrements  were  also  observed  in  other  measures, 
e.g.,  numbers  of  crashes  and  proximity  warnings,  from  Session  1 
(and  2)  and  Session  3  (and  4)  (Table  3) .  Both  the  average  num¬ 
ber  of  crashes  and  proximity  warnings  were  significantly  higher 
in  Sessions  3  and  4  than  in  Sessions  1  and  2,  averaged  over  all 
participants.  This  was  likely  because  Sessions  3  and  4  were  of 
longer  duration  and,  hence,  participants  had  more  opportunity 
to  experience  crashes  and  proximity  warnings.  However,  it  was 
also  interesting  to  note  that  the  number  of  crashes  and  proximity 
warnings  were  slightly  lower  in  Session  4,  as  compared  to  Session 
3.  This  may  have  been  due  to  some  sort  of  recovery  effect,  such 
as  described  in  the  previous  paragraph. 

EEG  Results  From  Spatial  Analysis 

To  spatially  represent  the  change  in  power  spectra  over 
different  segments,  we  used  scalp  maps  and  three  shades  of 
colors  (Fig.  6) .  All  changes  are  referenced  to  the  first  1 0-minute 
segment.  Areas  marked  in  blue  indicate  a  significant  increase 


in  EEG  power  spectra.  Areas  marked  in  red  show  a  statistical 
decrease,  and  areas  marked  in  green  represent  no  statisti¬ 
cally  significant  change.  Notice  that  the  blue  areas  follow 
an  increasing  pattern  across  time,  and  thus  are  thought  to 
represent  a  time-on-task  effect.  Areas  marked  in  red  do  not 
exhibit  a  consistent  trend  across  time  and,  thus,  are  thought 
not  to  be  a  useful  marker  for  a  time-on-task  fatigue  effect. 
Because  of  this,  we  focus  subsequent  discussion  only  on  the 
blue  regions. 

The  results  from  the  non-parametric  analysis  suggest  sig¬ 
nificant  and  consistent  changes  over  time  in  power  spectra  in 
theta,  alpha,  and  beta  frequency  bands,  as  shown  in  Figure 
6.  These  power  spectral  changes  were  observed  by  comparing 
the  first  10-minute  segment  to  the  subsequent  10-minute 
segments  in  sequence,  which  were  localized  to  frontal  mid¬ 
line  and  parietal  areas.  Further,  to  illustrate  the  consistency 
of  these  results  across  sessions,  we  have  presented  the  alpha 
power  spectral  plots  obtained  from  all  four  sessions. 
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Figure  7.  Scalp  maps  plotted  out  from  the  non-parametric  statistical  test  results  showing  significant 
changes  in  EEG  alpha  power  spectra  over  time  on  obtained  from  session  1  (a),  session  2  (b),  session  3 
(c),  and  session  4  (d)  comparing  the  first  10-minute  segment  with  subsequent  segments,  and  resulted 
significance  levels  are  color  coded  and  displayed.  Red  indicates  a  significant  decrease,  blue  indicates  a 
significant  increase,  and  green  means  no  significant  change. 


Figure  7  shows  the  scalp  maps  that  indicated  a  significant 
change  in  alpha  power  spectra  across  all  sessions  from  Partici¬ 
pant  9.  A  similar  pattern  was  also  observed  for  theta  and  beta 
bands  that  are  not  shown  here.  As  stated  earlier,  a  significantly 
increasing  pattern  in  alpha  band  was  observed  over  time  to  be 
localized  within  the  frontal  and  parietal  areas  along  the  midline 
for  Sessions  2,  3,  and  4.  Session  1  showed  random  changes 
similar  to  those  appearing  at  the  beginning  of  the  longer  ses¬ 
sions,  which  are  thought  to  be  the  result  of  temporary  changes 


in  brain  activity  not  related  to  the  time-on-task  effect.  Due 
to  this  observation,  the  performance  data  from  Session  1  (the 
30-minute  activity)  was  not  used  to  evaluate  the  time-on-task 
effect.  Instead,  performance  data  from  Sessions  2,  3,  and  4  were 
used.  Results  from  the  non-parametric  analysis  for  varied  session 
lengths  provided  more  information  on  increasing  patterns  of 
power  spectra  with  time-on-task  and  identified  spatial  regions 
(along  the  midline  from  the  frontal  area  to  the  parietal  area  over 
the  head)  associated  with  fatigue. 
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Figure  8  (a).  Scalp  electrode  locations  selected  as  the  representative  channels  used  for  mental 
state  transition  time  computation,  (b)  Whisker  plots  of  EEG  alpha  power  spectra  used  to  identify 
the  transition  times  of  mental  states.  The  grey  bars  represent  the  time  durations  used  for 
impedance  check. 


Indicators  of  Mental  State  Transitions 

To  determine  mental  state  change,  we  selected  representa¬ 
tive  channels  from  the  spatial  areas  that  are  identified  from 
the  non-parametric  statistical  analysis  and  increased  EEG 
spectral  powers  over  the  alpha  band.  Figure  8(a)  shows  the 
selected  representative  channels,  indicated  by  red  dots  over 
the  scalp  map,  all  of  which  appear  on  the  midline  from  frontal 


area  to  parietal  area.  Figure  8(b)  shows  the  Whisker  plots  of 
absolute  power  data  in  a  resolution  of  10  minutes,  obtained 
from  the  representative  channels  of  all  participants  in  Ses¬ 
sion  4.  The  grey  bars  after  the  60-minute  mark  represent  the 
impedance  check  interval,  during  which  no  EEG  data  were 
obtained.  From  these  Whisker  plots,  one  can  infer  that  there 
was  a  steady  increase  in  EEG  power  spectra  with  significant 
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Figure  9  (a).  Results  from  our  previous  study  showing  the  scalp  electrode  locations  for  representative 
channels  used  indicated  by  red  dots,  (b)  Whisker  plots  of  EEG  alpha  power  spectra  used  to  identify  the 
transition  times  of  mental  states. 


changes  occurring  from  60  to  80  minutes,  which  varied  within 
different  sessions  and  among  participants.  In  general,  most 
participants  showed  this  increasing  pattern  in  alpha  power 
spectra.  However,  there  were  individual  variations,  such  as 
observed  with  Participant  8.  Despite  individual  difference  in 


variation,  the  overall  pattern  of  increasing  EEG  power  spectra 
and  spatial  distribution  patterns  were  consistent  with  the 
results  from  our  previous  study  (Dasari,  Crowe,  Ling,  Zhu, 
&  Ding,  2010  Fig.  9),  thus  suggesting  stable  results  across 
multiple  studies. 
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Figure  10  (a).  Increasing  patterns  of  median  EEG  power  spectra  over  time  from  alpha,  beta,  and  theta 
bands  from  session  4  (a)  and  session  3  (b)  data  with  significant  changes  occurring  around  60  -  80 
minutes.  Black  lines  represent  the  transition  time  (TT)  for  mental  states. 


Figure  1 0  shows  the  median  values  of  absolute  spectral  power 
data  for  every  10-minute  segment  from  Sessions  3  and  4.  We 
estimated  the  transition  time  (TT),  defined  as  the  indicator 
for  mental  state  shift,  as  discussed  above,  using  the  following 
procedures.  First,  from  these  plots,  it  can  be  observed  there  was 
a  prominent  spectral  power  shift  occurring  between  50  and  90 
minutes  in  most  participants.  We  broadened  this  window  to  be 
from  40  to  100  minutes  and  searched  for  the  maximal  changes 
between  neighboring  segments  in  this  range  as  TTs,  which  are 
marked  by  the  vertical  lines  in  Figure  10  for  different  partici¬ 
pants.  This  prominent  shift  in  EEG  powers  was  identified  and 


computed  as  TTs  for  all  participants  and  sessions  (summarized  in 
Table  4).  Observe  that  most  transition  times  are  found  between 
60  to  80  minutes. 


Correlation  Analysis 

The  neurophysiological  pattern  identified  in  EEG  to  represent 
the  time-on-task  effect  (or  mental  fatigue)  depicts  a  visually  ob¬ 
served  correlation  with  cognitive  functional  decrements  observed 
in  C-Team  task  performance.  Fluctuations  in  performance, 
represented  by  response  time  and  routing  time,  correspond  with 
transition  times  identified  from  EEG  spectral  power  data  over  time. 


Table  4.  Summary  of  the  mental  state  transition  time  identified  from  EEG  spectral  power  data. 


Participant 

PI 

P2 

P3  P4 

P5 

P6 

P7  P8 

P9 

P10 

Session  2 

_ 

60 

Mental  State  Transition  Time  (minutes) 

60  60 

60 

60 

Session  3 

— 

60 

75  60 

60 

— 

74  77 

60 

50 

Session  4 

73 

77 

86  60 

80 

94 

60  50 

80 

60 

Note:  The  blank  fields  indicate  no  significant  change  observed  for  those  participant  sessions. 
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Figure  11.  Correlation  of  reaction  times  with  spatial  patterns  showing  significant  changes  in  EEG  alpha 
power  spectra  localized  to  frontal  midline  region  of  the  brain,  from  Participant  10,  session  4. 

Figure  1 1  shows  an  example  of  the  visually  observed  correlation  between  response  time  and  the  spatial  maps 
obtained  using  the  non-parametric  method  from  Participant  10,  Session  4.  The  performance  decrements  are  well- 
captured  by  physiological  patterns  from  EEG  at  both  60  and  100  minutes  where  the  frontal  midline  region  of  the 
brain  exhibits  increased  alpha  EEG  powers  as  compared  to  neighboring  segment  data  (i.e.,  deep  blue  color  means 
more  changes  than  light  blue  color). 
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Figure  12.  Correlation  of  alpha  median  power  obtained  from  EEG  power  spectral  analysis,  with  reaction 
time  performance  over  time  from  Participant  2,  Session  4. 


Figure  12  shows  an  example  from  Participant  2,  Session  4,  in  which  the  mean  response  time  is  shown  with 
the  median  alpha  EEG  powers  over  time.  The  plots  show  that  slow  response  times  can  be  visually  correlated  to 
increasing  patterns  of  EEG  alpha  power.  The  EEG  alpha  power  pattern  also  follows  the  response  time  pattern  when 
fast  response  times  appear  at  the  beginning  of  the  session  (due  to  the  alertness)  and  later  in  the  ninth  1 0-minute 
segment  (due  to  the  temporary  recovery  from  mental  fatigue).  The  improved  response  times  at  the  end  of  session 
may  be  attributed  to  the  expectation  of  finishing  the  task  soon.  A  similar  result  is  also  observed  in  all  participants 
and  sessions.  In  some  participants,  such  phenomena  are  also  observed  to  be  localized  within  the  parietal  region. 
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Figure  13.  Distributions  of  number  of  participants  showing  significant  changes  in  reaction  time  (green  bar), 
routing  time  (blue  bar),  and  mental  state  transition  times  (red  bar). 


For  the  comparison  at  the  group  level,  Figure  13  shows  the 
histogram  of  the  number  of  participants  in  sessions  3  and/or  4 
(combined)  that  displayed  a  significant  departure  from  baseline 
response  times  and  routing  times.  Also  included  in  the  histogram 
are  TTs  from  all  participants.  Each  bar  represents  the  number 
of  participants  falling  under  the  corresponding  1 0-minute  time 
window,  where  each  red  bar  represents TT  from  EEG.  Each  green 
bar  shows  the  significant  response  time  change,  and  each  blue 
bar  indicates  a  significant  routing  time  change.  Notice  that  the 
histograms  for  TT,  response  time,  and  routing  time  are  mostly 
clustered  in  the  time  window  from  50  to  90  minutes  and  peak 
between  60  to  70  minutes.  Since  the  time  markers  for  signifi¬ 
cant  changes  in  response  time,  routing  time,  and  power  spectral 
changes  were  mostly  found  after  60  minutes,  data  from  Sessions 
1  and  2  were  not  considered  for  this  correlation  analysis. 

DISCUSSION 

In  this  study,  we  analyzed  the  association  of  EEG  patterns 
related  to  mental  fatigue,  with  deteriorating  cognitive  perfor¬ 
mance  over  time  while  participants  performed  a  simulated  air 
traffic  control  task.  The  low-fidelity  air  traffic  control  task  used 
for  the  present  study  provided  a  more  realistic  approach  in  as¬ 
sessing  performance  changes,  as  the  task  scenarios  mimicked 
some  of  the  real-world  air  traffic  control  tasks.  Furthermore, 
the  nature  of  observations  of  the  time-on-task  effect  we  made 
in  the  present  study  was  more  spontaneous  and  realistic, 
compared  to  the  monotonous  single-event  tasks  used  in  other 


studies  (Kramer,  Sirevaag,  &  Huges,  1988;  Murataetah,  2005; 
Trejo  &  Mullane,  1995). 

The  realistic  nature  of  the  task  gives  us  a  better  understand¬ 
ing  of  mental  fatigue  and  its  impact  on  cognitive  performance. 
For  instance,  response  time  (i.e.,  aircraft  activation  time)  in  the 
C-Team  task  represents  the  level  of  alertness  in  monitoring  the 
appearance  of  aircraft,  while  the  routing  time  reflects  the  plan¬ 
ning  and  decision-making  activities  necessary  for  performing  air 
traffic  control  tasks.  Response  time  and  routing  time  obtained 
from  task  performance  data  in  the  present  study  show  signifi¬ 
cant  change  attributed  to  the  effects  of  time-on-task  fatigue  that 
mostly  occurred  around  60  to  80  minutes. 

EEG  spectral  power  studies  show  significant  alpha  and  theta 
activity  changes  over  time  that  are  believed  to  be  associated  with 
cognitive  functional  changes  (i.e.,  mental  state  changes).  Non- 
parametric  statistical  tests  implemented  on  EEG  data  show  that 
these  significant  changes  are  localized  to  the  midline  areas  from 
the  frontal  lobe  to  the  parietal  lobe  of  the  brain.  The  concept  of 
mental  state  transition  was  introduced,  and  the  times  for  such 
transitions  are  used  to  indicate  the  time  markers  for  the  devel¬ 
opment  of  mental  fatigue,  or  the  time-on-task  effect.  Obtained 
transition  times  from  EEG,  analyzed  by  the  processes  discussed 
above,  indicated  prominent  changes  in  EEG  data  indicative  of 
mental  state  that  occurred  approximately  around  70  minutes. 
These  patterns  were  consistent  in  sessions  from  the  same  par¬ 
ticipants  and  also  consistent  from  different  participants.  It  is 
noted  that  the  way  to  identify  transition  times  study  is  through 
the  visual  inspection  of  general  patterns  among  all  participants, 
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which  was  not  robust  and  optimal.  More  quantitative  measures 
such  as  medians,  changes  in  medians,  variances,  and  changes  in 
variances  of  EEG  power  spectra  in  a  finer  resolution  (e.g.,  5  min¬ 
utes)  can  be  further  explored  to  develop  more  robust  measures. 

Individual  and  group  analyses  of  mental  transition  times  and 
performance  changes  produced  meaningful  visual  correlations. 
Mental  state  transition  times  were  well  correlated  with  the  slow 
response  times  and  increased  routing  times.  Such  results  suggest 
that  patterns  obtained  from  EEG  data  are  reliable  indicators  for  the 
development  of  mental  fatigue  and  associated  cognitive  functional 
decrements.  In  many  examples,  EEG  rhythmic  powers  in  alpha 
bands  (also  in  theta  and  beta  bands)  fluctuate  with  the  performance 
data  (Figures  1 1  and  12),  which  suggests  that  EEG  rhythmic  pow¬ 
ers  are  promising,  reliable  indicators  in  real-time  monitoring  and 
can  even  be  used  to  predict  human  performance. 

During  our  study,  we  also  observed  the  mental  effort  deployed 
by  participants  to  momentarily  improve  their  performance  after 
first  experiencing  a  reduction  in  performance  due  to  the  onset  of 
mental  fatigue.  Examples  for  Participant  8  are  illustrated  in  Figures 
1 1  and  12,  as  well  as  in  Figure  8(a).  After  the  identified  mental  state 
transition  times,  the  performance  of  participants  show  large  fluctua¬ 
tions.  The  initial  recovery  in  performance  during  these  fluctuations 
suggests  that  the  individual  may  be  attempting  to  counteract  the 
mental  fatigue  effects  by  increasing  mental  effort.  Flowever,  the 
increase  in  mental  effort  appeared  to  be  insufficient  to  maintain 
the  desired  performance  level  and  was  followed  by  a  subsequent 
decline  in  performance.  It  is  possible  that  multiple  episodes  of 
recovery  from  mental  fatigue  may  be  observed  under  more  taxing 
conditions  from  those  encountered  in  the  experiment.  Even  though 
this  is  speculation,  we  are  certain  that  future  researchers  can  take 
advantage  of  this  work  and  proceed  forward  to  understand  how 
increases  in  mental  effort  made  after  onset  of  mental  fatigue  would 
affect  brain  functions  and  subsequent  behavioral  performance. 

One  limitation  that  we  experienced  during  this  study  was  the 
impedance  check  at  the  60-minute  time  mark.  We  conducted 
the  impedance  check  to  make  sure  the  skin-electrode  interface 
impedance  was  lower  than  50  KO  in  order  to  maintain  a  high 
signal-noise-ratio  in  recordings.  Due  to  this  impedance  check, 
10  minutes  of  data  were  lost  at  the  time  (i.e.,  60  minutes),  which 
is  around  the  identified  moment  for  the  critical  development  of 
mental  fatigue.  Furthermore,  the  impedance  check  also  impacted 
the  alertness  level  of  participants  since  they  were  presented  with  a 
new  event  that  was  not  related  to  the  task,  even  while  instructions 
had  been  given  to  participants  to  ignore  the  impedance  check.  To 
remove  artifacts,  a  post-recording  signal  processing  technique,  i.e., 
ICA,  was  used  (Makeig  et  al.,  1996).  An  alternative  possible  solu¬ 
tion  to  this  issue  is  the  use  of  dry  electrodes  for  long  hours  of  EEG 
recording  (Justin,  Jason,  James,  &  Glenn,  2009).  Furthermore, 
the  MRAB  results  in  the  present  study  did  not  show  consistent 
differences  among  pre-task  and  post-task  cognitive  tests,  which 
suggests  that  MRAB  was  possibly  not  sensitive  in  identifying  the 
time-on-task  effect.  Other  possible  reasons  for  this  could  have 
been  the  short  nature  of  MRAB  tasks  and  the  variety  of  small 
tasks  in  MRAB  that  may  help  participants  become  engaged  after 
performing  long,  monotonous  C-Team  tasks. 


Finally,  the  present  study  serves  as  a  template  for  conducting 
high-fidelity  simulation  experiments  involving  actual  air  traffic 
control  participants.  We  demonstrated  that  mental  state  transi¬ 
tions  related  to  time  on  task  fatigue  could  be  identified  while 
participants  performed  low-  fidelity  aircraft  separation  tasks 
involving  changes  in  speed,  heading  and  altitude.  The  regions 
of  the  brain  associated  with  performing  these  tasks  are  likely  to 
be  the  same  regions  activated  during  actual  air  traffic  control 
operations.  However,  the  intensity  and  duration  of  mental  state 
transitions  are  likely  to  be  different.  Therefore,  before  any  gen¬ 
eralization  can  be  made  to  the  air  traffic  control  population,  it 
will  be  important  to  replicate  this  study  under  high  fidelity  air 
traffic  control  simulation  conditions. 
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